Importance:
- Reproduction and critical assessment
- Erroneous results and misinformation
- Standardization
Many aspects already studied in (among others) psychology, but not in sociology.
07-12-2021
Importance:
Many aspects already studied in (among others) psychology, but not in sociology.
Note that not all topics are (fully) addressed in this presentation.
For JMF, statcheck could initially not extract results with a minus sign. One of the package’s authors solved this problem for us by adapting the GitHub version of the package, which we installed using the R code below.
install.packages("devtools")
devtools::install_github("MicheleNuijten/statcheck")
install.packages("statcheck")
\[\scriptsize \begin{align} H1: log(Inconsistency_{i}) &= \beta_0 + \beta_1\times hypothesis_i + \epsilon_i \\ H2: log(Gross\,inconsistency_{i}) &= \beta_0 + \beta_1\times hypothesis_i + \epsilon_i \\ H3: log(Publication\,bias_{i}) &= \beta_0 + \beta_1\times hypothesis_i + \epsilon_i \\ H4: log(Marginal\,significance_{i}) &= \beta_0 + \beta_1\times hypothesis_i + \epsilon_i \end{align} \]
13 of 143 journals (9.1%) requested authors to adhere to APA statistical reporting guidelines
APA (n = 505):
- 69 results (13.7%) were inconsistent
- 8 results (1.6%) were grossly inconsistent
Hyp (n = 351):
- 17 results (4.8%) were inconsistent
- 3 results (0.9%) were grossly inconsistent
Clarivate Analytics’ Web of Science. 2016. “Journal Citation Reports: Sociology, 2016.” com.proxy.library.uu.nl/JCRJournalHomeAction.
Epskamp, Sacha, and Michele B. Nuijten. 2016. Statcheck: Extract Statistics from Articles and Recompute P Values. https://cran.r-project.org/src/contrib/Archive/statcheck/.
Gerber, Alan S., and Neil Malhotra. 2008. “Publication Bias in Empirical Sociological Research: Do Arbitrary Significance Levels Distort Published Results?” Sociological Methods and Research 37 (1): 3–30.